{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 皮马印第安人糖尿病预测——SVM\n",
    "\n",
    "Pima Indians Diabetes Data Set（皮马印第安人糖尿病数据集）是根据现有的医疗信息预测5年内皮马印第安人糖尿病发作的概率。\n",
    "\n",
    "数据链接：https://archive.ics.uci.edu/ml/datasets/Pima+Indians+Diabetes  \n",
    "类似的有，Kaggle-Practice Fusion Diabetes Classification任务：https://www.kaggle.com/c/pf2012-diabetes\n",
    "\n",
    "## 1、导入数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数据读取及基本处理\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "# 可视化数据信息\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sn\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pregnants</th>\n",
       "      <th>Plasma_glucose_concentration</th>\n",
       "      <th>blood_pressure</th>\n",
       "      <th>Triceps_skin_fold_thickness</th>\n",
       "      <th>serum_insulin</th>\n",
       "      <th>BMI</th>\n",
       "      <th>Diabetes_pedigree_function</th>\n",
       "      <th>Age</th>\n",
       "      <th>Target</th>\n",
       "      <th>pregnants_log</th>\n",
       "      <th>Plasma_glucose_concentration_log</th>\n",
       "      <th>blood_pressure_log</th>\n",
       "      <th>Triceps_skin_fold_thickness_log</th>\n",
       "      <th>serum_insulin_log</th>\n",
       "      <th>BMI_log</th>\n",
       "      <th>Diabetes_pedigree_function_log</th>\n",
       "      <th>Age_log</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.639947</td>\n",
       "      <td>0.866045</td>\n",
       "      <td>-0.031990</td>\n",
       "      <td>0.670643</td>\n",
       "      <td>-0.181541</td>\n",
       "      <td>0.166619</td>\n",
       "      <td>0.468492</td>\n",
       "      <td>1.425995</td>\n",
       "      <td>1</td>\n",
       "      <td>0.825781</td>\n",
       "      <td>0.909473</td>\n",
       "      <td>0.051923</td>\n",
       "      <td>0.715064</td>\n",
       "      <td>0.018491</td>\n",
       "      <td>0.266621</td>\n",
       "      <td>0.612059</td>\n",
       "      <td>1.437767</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.844885</td>\n",
       "      <td>-1.205066</td>\n",
       "      <td>-0.528319</td>\n",
       "      <td>-0.012301</td>\n",
       "      <td>-0.181541</td>\n",
       "      <td>-0.852200</td>\n",
       "      <td>-0.365061</td>\n",
       "      <td>-0.190672</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.802604</td>\n",
       "      <td>-1.312334</td>\n",
       "      <td>-0.446834</td>\n",
       "      <td>0.133506</td>\n",
       "      <td>0.018491</td>\n",
       "      <td>-0.840901</td>\n",
       "      <td>-0.324994</td>\n",
       "      <td>-0.050575</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.233880</td>\n",
       "      <td>2.016662</td>\n",
       "      <td>-0.693761</td>\n",
       "      <td>-0.012301</td>\n",
       "      <td>-0.181541</td>\n",
       "      <td>-1.332500</td>\n",
       "      <td>0.604397</td>\n",
       "      <td>-0.105584</td>\n",
       "      <td>1</td>\n",
       "      <td>1.152449</td>\n",
       "      <td>1.762418</td>\n",
       "      <td>-0.623068</td>\n",
       "      <td>0.133506</td>\n",
       "      <td>0.018491</td>\n",
       "      <td>-1.464829</td>\n",
       "      <td>0.749586</td>\n",
       "      <td>0.047687</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.844885</td>\n",
       "      <td>-1.073567</td>\n",
       "      <td>-0.528319</td>\n",
       "      <td>-0.695245</td>\n",
       "      <td>-0.540642</td>\n",
       "      <td>-0.633881</td>\n",
       "      <td>-0.920763</td>\n",
       "      <td>-1.041549</td>\n",
       "      <td>0</td>\n",
       "      <td>-0.802604</td>\n",
       "      <td>-1.128548</td>\n",
       "      <td>-0.446834</td>\n",
       "      <td>-0.578264</td>\n",
       "      <td>-0.552520</td>\n",
       "      <td>-0.581597</td>\n",
       "      <td>-1.063014</td>\n",
       "      <td>-1.247065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-1.141852</td>\n",
       "      <td>0.504422</td>\n",
       "      <td>-2.679076</td>\n",
       "      <td>0.670643</td>\n",
       "      <td>0.316566</td>\n",
       "      <td>1.549303</td>\n",
       "      <td>5.484909</td>\n",
       "      <td>-0.020496</td>\n",
       "      <td>1</td>\n",
       "      <td>-1.703581</td>\n",
       "      <td>0.599436</td>\n",
       "      <td>-3.302832</td>\n",
       "      <td>0.715064</td>\n",
       "      <td>0.612171</td>\n",
       "      <td>1.455322</td>\n",
       "      <td>4.158488</td>\n",
       "      <td>0.143015</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pregnants  Plasma_glucose_concentration  blood_pressure  \\\n",
       "0   0.639947                      0.866045       -0.031990   \n",
       "1  -0.844885                     -1.205066       -0.528319   \n",
       "2   1.233880                      2.016662       -0.693761   \n",
       "3  -0.844885                     -1.073567       -0.528319   \n",
       "4  -1.141852                      0.504422       -2.679076   \n",
       "\n",
       "   Triceps_skin_fold_thickness  serum_insulin       BMI  \\\n",
       "0                     0.670643      -0.181541  0.166619   \n",
       "1                    -0.012301      -0.181541 -0.852200   \n",
       "2                    -0.012301      -0.181541 -1.332500   \n",
       "3                    -0.695245      -0.540642 -0.633881   \n",
       "4                     0.670643       0.316566  1.549303   \n",
       "\n",
       "   Diabetes_pedigree_function       Age  Target  pregnants_log  \\\n",
       "0                    0.468492  1.425995       1       0.825781   \n",
       "1                   -0.365061 -0.190672       0      -0.802604   \n",
       "2                    0.604397 -0.105584       1       1.152449   \n",
       "3                   -0.920763 -1.041549       0      -0.802604   \n",
       "4                    5.484909 -0.020496       1      -1.703581   \n",
       "\n",
       "   Plasma_glucose_concentration_log  blood_pressure_log  \\\n",
       "0                          0.909473            0.051923   \n",
       "1                         -1.312334           -0.446834   \n",
       "2                          1.762418           -0.623068   \n",
       "3                         -1.128548           -0.446834   \n",
       "4                          0.599436           -3.302832   \n",
       "\n",
       "   Triceps_skin_fold_thickness_log  serum_insulin_log   BMI_log  \\\n",
       "0                         0.715064           0.018491  0.266621   \n",
       "1                         0.133506           0.018491 -0.840901   \n",
       "2                         0.133506           0.018491 -1.464829   \n",
       "3                        -0.578264          -0.552520 -0.581597   \n",
       "4                         0.715064           0.612171  1.455322   \n",
       "\n",
       "   Diabetes_pedigree_function_log   Age_log  \n",
       "0                        0.612059  1.437767  \n",
       "1                       -0.324994 -0.050575  \n",
       "2                        0.749586  0.047687  \n",
       "3                       -1.063014 -1.247065  \n",
       "4                        4.158488  0.143015  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dpath = './Pima_data/'\n",
    "\n",
    "# 采用原始特征 + org特征\n",
    "train1 = pd.read_csv(dpath +\"FE_pima-indians-diabetes_org.csv\")\n",
    "train2 = pd.read_csv(dpath +\"FE_pima-indians-diabetes_log.csv\")\n",
    "\n",
    "# 去掉多余的id\n",
    "train2 = train2.drop([\"Target\"], axis=1)\n",
    "train =  pd.concat([train1, train2], axis = 1, ignore_index=False)\n",
    "\n",
    "del train1\n",
    "del train2\n",
    "\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "（样本数目，特征维数）=  (768, 17)\n"
     ]
    }
   ],
   "source": [
    "print(\"（样本数目，特征维数）= \", train.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2、构建模型\n",
    "\n",
    "### 2.1、线性SVM\n",
    "\n",
    "#### （1）分割数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.sparse import csr_matrix\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "y = train['Target']\n",
    "X = train.drop([\"Target\"], axis=1)\n",
    "\n",
    "# 保存特征名字\n",
    "feat_names = X.columns \n",
    "\n",
    "# 稀疏矩阵\n",
    "X = csr_matrix(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （2）网格搜索找最佳参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7721354166666666\n",
      "{'C': 1}\n"
     ]
    }
   ],
   "source": [
    "# 线性SVM\n",
    "from sklearn.svm import LinearSVC\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "tuned_parameters = { 'C': [0.001, 0.01, 0.1, 1, 10, 100, 1000] }\n",
    "\n",
    "svc_lr = LinearSVC(penalty='l2', loss='squared_hinge', max_iter=5000)\n",
    "grid = GridSearchCV( svc_lr, tuned_parameters, cv=5, return_train_score=True)\n",
    "\n",
    "grid.fit(X, y)\n",
    "\n",
    "print(grid.best_score_)\n",
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （3）评价指标曲线图\n",
    "\n",
    "    不同正则参数C下，对应的正确率曲线"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEICAYAAABF82P+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl4VPXVwPHvEUJYBJQQFlEW64YrSPRFVARBkEEFcamWVrEqpSpKteJefS0iaq0Kr1WBtmoVFRcsCEhFNhdEA6KIirgiCohQ2ZdAzvvHmZgQhmSSzJ07Mzmf55lnMnOXORPIPfe3i6rinHPOlbZX2AE455xLTZ4gnHPOxeQJwjnnXEyeIJxzzsXkCcI551xMniCcc87F5AnCOedcTJ4gnHPOxeQJwjnnXEw1ww6gKho3bqytW7cOOwznnEsr8+fP/1FVc8vbL60TROvWrcnPzw87DOecSysi8k08+3kVk3POuZg8QTjnnIvJE4RzzrmYPEE455yLyROEc865mDxBOOeci8kThHNuz3zFyWotrcdBOOcqads2WLUKVqywx8qVsZ9XrYKsLMjJgcaN7RHPz3XqhP0NXQJ4gnAuU6jCunVlX/CLfl67dvfjRSA3F5o1g+bN4fDD7ecdO2DNGvjxR3t89ZW9/u9/9xxL3bqeVDKAJwjnUt2OHfDDD2Vf8Iuet27d/fjsbLvgN28Ohx4KXboUJ4Gi5+bNLTlkZVUsrrVrd00eJX8u+dqTSlryBOFcWDZuLP+Cv2IFrF4duy2gUaPiC/yJJxZf6Etf/Bs2tNJBotWsCU2a2CNeRUmldDKJ9fNXX9nzTz/t+XwVSSpFvxMXN08QziXaxo3w5Zd7vuAX/bxx4+7H1qxZfGFv2RL+539i3+03bWolg3QTVFIpel1eUnn9dTj11MR8l2rAE4RzVbF6Nbz//q6PpUt3v+Nv0KD4At+hw57v9hs1gr28c+EuqppUih4XXQQvveQJogI8QTgXD1VYtgwWLNg1GXz3XfE+rVpB+/bQvz+0bQv77Vd8t1+vXnixV0exksoTT8DkyTBqVDBVbhnIE4Rzpe3cCUuW7F4yKGpg3WsvOOwwa+xt394e7drZ3b9LXZEITJwIn35qCdyVyxOEq962boWPPrIEUFQ6+PBD2LLFtmdnw1FHwbnnFieDo4+2xlGXXnr1sucpUzxBxCmwBCEiw4GTAAVuUtW3S2x7CSi63ToQuB2YAPwdaAZsAv6gqouDis9VQ+vWwcKFu5YKPv7YSgxg7QTt2sHvflecDA47rGJdP13qatnSkv2UKXDddWFHkxYCSRAi0g2oq6qdRaQhMEFEeqpqAYCq9ovuVwN4CXgS6AksVNU/i8j+wAPAeUHE56qBlSt3ryL64ovi7c2aWQI488ziZNCmjTcQZ7pIBP76V1i/3m4IXJmCKkF0A8YCqOo6EZkNHAUsKLXfAGCcqu4UkanAUBG5BNgfuDXWiUVkIDAQoGXLlsFE79KHqnVtLFlF9P77liCKHHigJYBLLilOBs2bhxezC08kAvfcY91dzz477GhSXlAJIgdYVeL1SmCXPmoikg30iT4ArgZeVtUHRaQpVvW0G1UdDYwGyMvL85nEqpMdO+CTT3YtFSxcaFVHADVqWN1yjx67Nh43bBhu3C51nHCC/X+YMsUTRByCShBrgFxgdfR1M+C9UvsMAsao/txh/FTgbABVXSUifxKRj1R1Q0AxulS2eTMsWlScCBYssNfbttn2OnWssfjCC4uTwZFH+tQLrmxZWXYDMWWKlT69u2uZgkoQM7Dqo6HRNojOwPCijSJSDzhVVR8qccynwCnAzOgxJwLbA4rPhWXHDqv/Xbdu18f69VYtVNSI/OmnUFhox+yzjyWAK6+EY4+1nw85xPq6O1dRkQg8/7z1VjvmmLCjSWmB/IWp6nQR6R5tewC4CcgTkfaq+jdgCDCy1GHDgcdE5E9AbeAWVd0WRHyuElTt7j3Whb2s16Xf27y57M9p0cISwDnnFJcMWrXyOz2XOKefbs9TpniCKIdoCAuCiMgRiejCmpeXp/n5+YkIKbOp2rw/Vbmwr1sHBQXlf1a9elbH26CBPRc9Sr+O9V5Ojg82c8mRl2fVkW+8EXYkoRCR+aqaV95+oZTRfXxDgmzdCv/4h00BUdaFfcOG4uqaPdlrr+ILdtFz8+Y2DiCei3uDBvbwah+XDiIRuOsum6/Jb0r2yP+a09XChfCb39go4Fq1dr9w/+IXFbuT33tvr8Zx1UckAn/+M/znP3DBBWFHk7I8QaSbnTvh3nvh9tutSmbKlOIpBJxz8TnuOFsnYsoUTxBl8ASRTj7/HC6+GN5+G847Dx55xJKEc65iatSwxuqpU6361UfQx+S/lXSgCo89ZoO+Fi+Gp56C557z5OBcVUQitk6Ed3TZI08QqW7FCjjjDBg0CDp2tMFi/ft7e4FzVdWjh5UcpkwJO5KU5Qkilb3wgs0+OWMGjBxpDWoHHBB2VM5lhpwcu+nyBLFHniBS0U8/wa9/be0MbdrYyOLBg72e1LlEi0Tgvfdg1ary962G/IqTal5/3UoNzz4Ld9xhDdKHHRZ2VM5lpkjEnqdNCzeOFOUJIlVs2QLXXAPdu9to5LlzrSurL1bjXHDatbMBoZMnhx1JSvIEkQry820SupEjrSppwQLrp+2cC5aIjSOaNs0mknS78AQRph074M47bY76DRusEXrkSF/v2Llk6t3bpqSZOzfsSFKOJ4iwLFkCJ55o1Ujnn2/dV087LeyonKt+une3OcS8N9NuPEEkmyo8/LBNY710qQ14e/pp2HffsCNzrnpq0ABOPtkTRAyeIJLpu++gZ0+46io45RSbaO/888OOyjkXidgCQsuXhx1JSvEEkSzPPGNLYr71ls2hNGUK7Ldf2FE556C4u+vUqeHGkWI8QQRt7VqbLfJXv7LxDAsX2rQZPlWGc6mjbVtbudCrmXbhCSJI06bZoLcXX4Rhw2z1qoMPDjsqF6CPPoKzz7YZHLZuDTsaFzcRK0VMn25L6zrAE0QwNm2CK6+06YT32QfmzYNbbvHV1jLYN9/AgAFw9NEwc6YNY6ld27aNGGE1Fzt3hhqiK08kYkvzVtNlSGPxBJFo77xjPZQeeQSuvRbmz7dBcC5jzZ8Phxxis6Ncdx18+SWMGmXbNm+Ghx6ya0+rVnDrrfDFF+HG6/aga1fIzvZqphI8QSRKQQHcdpuNbdi2zWZgvf/+4ttIl1E2bbKCIdhsDddfb72W77tv1yWO69a10sULL8Axx8Ddd8NBB8ETT4QTtytDvXrQpYsniBICSxAiMlxE5ojIbBHpVGrbSyIyK/pYJiKXiMhZJd6bIyKfi0h61Ml8/LFVOg8bZutEf/ih/UdzGaegwNZuOvhgKxVs2mSLkw0btueZ2GvVgnPOsel+li2Du+6Cbt1s28SJ8LvfWbJRTd73cHvQu7cNYvVinlHVhD+AbsCD0Z8bAjOArBj71QD+DdQo9f5vgF+X9zkdOnTQUO3cqfrAA6rZ2aqNG6u+9FK48bjAFBaqPv+86iGHqILqiSeqvvlm1c97//2qderYOY84wl6vWlX187pKWrrU/jFGjQo7kkAB+RrHtTyoEkQ3YGw0Aa0DZgNHxdhvADBOVX9uvhORLOA8YFxAsSXGsmU2RP8Pf7ApMhYtsu4rLiPl59vyHDVrwr//be2YJ55Y9fNeey2sXAmjR0P9+taG0b178XYvVSTZQQdZg5JXMwHBVTHlACVX4FgJNCm5g4hkA32A8aWOvQx4XFULY51YRAaKSL6I5K9evTqBIcdJFZ580rqvvvcejB1r9QTNmiU/FheoDz6wCzdYr6SpU6328KyzEjuMpUEDuPxymytu8WKbrxGsgfvII+Hmm619wyVJJGJd0TZvDjuS0AWVINYAuSVeNwN+KLXPIGBMtLgDgIjUAXoBE/Z0YlUdrap5qpqXm5u7p92C8eOPcO65cPHF1p/xgw/g0kt90FuG+eorW9CvfXv405+KrxOnn27tDUE6/PDi5qs1a+AXv4B777Wb2s6d4fHHrd3DBSgSsUEss2aFHUnogkoQM7DqI0SkIdAZWFS0UUTqAaeq6qRSx10F/K1k0kgZkyfb7dwrr8A999h/ngMPDDsql0Br1sCQIXDooTa2cehQ+OST8GZfP+AAK5wuW2ZjKVatgksusZjAEkUK/qWkv86d7R/dq5mCSRCqOh0oFJHZwETgNiBPRK6I7jIEGFnyGBFpAJyoqq8GEVOlbdwIAwfCGWdAkyZWrTR0aPC3ki7p/vtf66E0YAB8/rldlFNhkt399oMbboBPP4V334UOHez9a66xEsd99/mSygmVnW0NQZMnewaOpyU70Q/giBjv1QXaVOQ8gfdievNN1QMPVBVRveEG1a1bg/28CtqxQ3XRIuth4ypu+3bVhx9WvfTS4vdWrw4vnop66inVTp2s002NGqp9+qhOmxZ2VBni0UftF/vxx2FHEghC7sVUXlJaHOO9zar6VRjx7GbbNrjpJitqqsKcOXY7mZ0ddmRs3mzVDpdeaneW555b3AQyY4b1iHFlKyy0ZTjatrUZUZYuLW5naNw43Ngqon9/mxz444+tN9Q771gNKNh/288+Cze+tNarlz1X92qmeLJIqj4CKUF8+KHq0Ufb3cNll6muX5/4z6ike+4p7jPfoIHqBRcU3zFu367asKEVdjp2VB0+XHXxYi9dlPbJJ6rHHmu/w6OOUp08OXN+R9u3q/73v/bzrFn2HTt1Uv3731Pqv3H6OPJI1VNPDTuKQJDKJYiUtHOnVebm5dlt+MSJMGaMdU5PMlWrb77nHutr/+WX9v6hh1rJ4bXXYPVqW2KiRw/bVrOmFXTuvNOWur75ZjjiCBg+vPjrVec12bdsseemTe33+8QT8P771mElUzqhZWXZ3JBgbRP33muzzV96KTRvDr/9rTXEuzj17m0DXtavDzuS8MSTRVL1kbASxJdfqp58st1ynX226g8/JOa8FbRiheof/6h68MEWCtjd7jvvVPxcy5erPvKIFYhUVWfMUG3USPU3v1F94YXqc0f5+edW0jrmGBv4rpo5JYZ4FBaqvv22FYYPPFB12zZ7/803Vb//PtzYUt7s2fZHmIEzJBBnCSL0i3xVHlVOEIWFVv7ee2+rs3niiaRePTZuVJ0wQfW11+z16tWqtWur9uhhjafffpu4z3r/fUsOjRrZv3qtWqq9eqmuXJm4z0glK1eqXnmlas2aVi13882qW7aEHVW4duyw58JC1datrWH7zDPt/+D27eHGlpKK6m0vuyzsSBLOE0R5Vq5UPess+xV06aL69deVP1cFrFihOmaM6hlnWDIA+yMtsmlTsJ9fUKA6c6bqH/6gmpdnr1Vt6plhwzKjV9R776nWq2cXwEGD/E45liVLVG+8UbV5c/s/2KSJ6pNPhh1VCjrvPNX99kv/P4pSPEGUZcIE1dxcm2Tvr38trnsIQGGh6jffFL8+6ST7rbdqpXr11aqvv54ad2/nn68/V2sdeKDqkCGqb70VdlTx27atuDpt+3bVwYPtIujKVlCgOmmS1axOnmzvffml6ujRquvWhRtbSvjnP+2PYuHCsCNJKE8Qe7J9u2rbtqrt26t+9FHFj49DQYFVX157reovfqGalaX600+27a23VD/4IDVvSL7/XvWxx1QjEcud/fsXb5s0KTXbLXbuVH36adU2bVSbNg2+BFYd/OUvdmWoW1f14oszdihAfFassF/G8OFhR5JQniDK8s03xa11CTZpkmpOjv5cz3/66dZYnIoX17Js2GAN3arWNbTo+/Tsqfq3vyW2faQyCgtVX31VtV07i61dO3udiok33RQWWseIyy9XrV/fEsW//hV2VCHq0MHmd88g8SaI6tnNtWVLW8WlilassNk+e/eGl1+29w4+2MbYPP+8ze03dSoMGhRKb9kq2XtvaNHCfj7oIOtCO3iwraNyxRU2T1DRGKKCguTPSDBvnk2et24dPP20LfvZs2fmdFkNkwj8z//Y/+0lS6zn98yZYUcVokjEptpduzbsSJIvniySqo8wFgzavl31rrtUjz9ef66zb9PGpj2oDgoLbQDe8OHFg7Luvdd6xQTdprJkieq4ccVxPPdcYAVBV0JBQXEPsI8/Vv3qq1DDSb65c+0P/Zlnwo4kYYizBCGa7Fu/BMrLy9P8/PxAP2PHDnjzTfj+e/jVrywlHHww5OTYugB9+tiAtOp85zp5Mjz6KEyfbrMk77OP/W4efzwxv5cVK2wA4JgxNnnesmVQp07Vz+sqRtVW1l26FP71Lys5Vws7d9p6L7162VowGUBE5qtqXnn7pceaz0m2YQO8+qoNpp482Wb5bNECLrzQLngffhjeFNCpqHdve2zaZKO8J060n4uSw9Ch0KqVJY09rdscy7p1Nrj9gQdg+3arqrvtNk8OYRGBceNs/q8zzrDR+nfeWQ0mNq5Rw+ozp061ibz2qkY18/EUM1L1kcgqpu++Kx5IdM01VqJs1Ej1ootUX3zRGm1dxW3bpnr44cXVce3bq95xhzV8l2fRIhvLcMEFtlSwSw2bN9sMuGBTFa1dG3ZESTBunH3hefPCjiQh8EbqsqlaSWDYMFtOskULa4cC+P3vYfZsm2P/iSegXz9rtHUVV6uWLaP5ySc2t1SdOvC//2s3Y2ClhOnTrYSwc6eV4K+7zrYdeaTNQ/XMM9ZQ7lJDnTq20u4//mGlinr1wo4oCXr0sJJDNZvdtVq2QSxdav/eX39trzt2tOqP3/wG9t8/sTG63a1aZYlj333hqafs996wIeTm2kI9HTrYHGlelZT6VC1JrFkD48dbNWDGtsedeKJ12Xv33bAjqbJ42yCqZQmiVSvrujd6tDWAzp1ryz94ckiOpk2LV2rr1w/+/W845xxrB3z2Wfv78+SQHoqSwZgx1v353HOtVJiRIhFbUbIaLd9XLUsQzrnEUrXOBEOHQps28MILcMwxYUeVYO+/D8cea93zLr447GiqxEsQzrmkEbFV7WbNsh5sHTtaT8CM0q6dLaxRjdohPEE45xLmpJPsRrtfP7vZzigiNhZi2rRqs/qWJwjnXEI1bWrTnzRpYtfRK66wKVoyQiRijSxFXR4zXMolCBHZS0SeEREfxOdcmvvsM+t40KGDdUZIe9272/q+1aSaKbAEISLDRWSOiMwWkU6ltr0kIrOij2UickmJzdcAr6hq9SjDOZfBDj8cFiywcSx9+8INN6R57UzDhnDyyZ4gqkJEugF1VbUzcBYwTESyiraraj9V7QJ0A94Hnowe1xo4VlWfDiIu51zytW5t85kNGgT33guXXhp2RFUUidgo2+XLw44kcEGVILoBYwFUdR0wGzgqxn4DgHGqujP6+mHgMBGZJCL7BRSbcy7JateGRx6xSf6uuSbsaKooErHnoukAMlhQCSIHKDmaZCXQpOQOIpIN9AHGR193Ar4FjgfuAO6PdWIRGSgi+SKSv3r16sRH7pwLzK9/Xdy76ZprYMQIm/8urbRta6Ntq0E1U1AJYg2QW+J1M+CHUvsMAsZo8Ui9E4CiZdPnAzHH0qrqaFXNU9W83NzcWLs451Lcjh2wcqXNYNC3r82YnDZErBQxfTps2xZ2NIEKKkHMwKqPEJGGQGdgUdFGEakHnKqqk0oc8wXQM7q9JZDOTVnOuTLUrGm9m0aOtAF1HTpYY3baiERg40abNCyDBZIgVHU6UCgis4GJwG1AnohcEd1lCDCy1GH/BnJEZA4wBrghiNicc6lBxJaxnTPH5sDr0cOuuWmha1fIzs74aqZQ5mISkSNUdXFVz+NzMTmXGX78ET74ALp1s3mdtm+3629KO/10mxL600/DjqTCUnoupkQkB+dc5mjc2JID2Mywxx1ng+xSWiQCS5Zk0DDx3aXcSGrnXPXWurWtAZ+XZ7PCpqxq0N01rgQhIqcEHYhzzoG1RSxYYKOwzzvPZoktKAg7qhgOOggOOSSj2yHiLUEcLyIvi8iFIuKlDudcoFq2tMbrwYNtnYk33ww7oj2IRGDmTNi8OexIAhHXxV5V7wPOBWoAL4nItSLiqzQ75wJTq5Z1g/3gA+s0BPDdd+HGtJtIBLZutYUwMlDcpYHo5HnrgUJgX+AfInJvUIE55xzA0Ufb8zvvwIEHwrBhKTT6unNnqFs3Y6uZ4m2DuEREJgLNgQtV9TZVPR/4V6DROedc1JFH2prXt90GZ54Ja9eGHRHWF7d7d5g82frnZph4SxAbgD6q+piq/jy2XFUXlXGMc84lzN57w1NPwd/+ZrNcHHsspMQwqEgkbcdDlCfeBFET6AcgIueKSJ/gQnLOudhE4Pe/t0ZrVVv9M3S9etlzBlYzxTWSWkTeALqo6s7oSm//UdVTA4+uHD6S2rnq66efoEED2GsvWwf7kEOgXr2QgjnqKFtj9fXXQwqgYhI9knpH0ZoN0cZqqUpwzjlXVfvsY8lh82a7iT/++BBreSIRm7hv/fqQAghGvAliu4jUAfDurc65VFK3Ljz9NKxebaOvn3suhCB697bRfGlSgohXvAniLuApEfk18CwwLLiQnHOuYrp1s2qmdu3gggvg6quTvPb1CSfYetUZ1g5RM56dVHWOiHyBrfZ2pap+E2xYzjlXMS1a2KDmG2+Eb7+FGjWS+OFZWTZHyJQp1noumVELX5GBct+p6gRPDs65VJWVBfffD888Y9fozz9PYk+nSMRmGfzwwyR9YPDiHSj3qoh8JyLTo8+ZO32hcy7tFZUebr/dGrBvvx127gz4Q08/3Z4nTw74g5In3hLEQuAcVe0OnA/MCy4k55xLjDFj4KKL4M47LVFs2BDghzVrZmunZlA7RNyzuarqOwCq+hZwcnAhOedcYtStC//8J4weDa+9Bn/5S8AfGInA3LkpMg9I1cWbIGqVep2V6ECccy4IInD55XD22fDuuwFPmRSJ2EyC//lPgB+SPPEmiJdF5C4R2V9EhgH/DjIo55xLtCeftNqfQDsYHXcc5ORkTDVTvOtB/AX4ABgKfKCq9wcalXPOJdjee1tyWL06wPV9atSwxuqpU1NoTvLKi7cX059VdbyqXq2qzwcdlHPOBWHZMlvzevToAD+kd2/48ccUmWq2auKtYtpXRC4QkaYi0kBEGpR3gIgMF5E5IjJbRDqV2vaSiMyKPpaJyCXR968UkfnR90dV4vs459wetWxptUD33QfbtpW/f6X06GGTRGVANVO8CaIe0BMYATwEPFjWziLSDairqp2Bs4BhIvJzw7aq9lPVLkA34H3gyeimDkA3Ve2iqoMr8kWccy4et9xi49meeCKgD8jJgY4dMyJBxDXVBnBHiZ+V8mdz7QaMBVDVdSIyGzgKWFBqvwHAuKKZYoFjgPEishG4WlWXxxmfc87FpXt3m/l1xAj47W+hZrxXwYqIRODWW2HVKmjaNIAPSI54SxBDgGuAm4A3gFvL2T8HWFXi9UqgSckdRCQb6AOML/F2T1XtAdwOxGwIF5GBIpIvIvmrV6+OM3znnDMiVor46iuboTsQkYg9v/pqQB+QHPH2YvqDql6rqoOAjpQ/DmINkFvidTPgh1L7DALGaIkVi1T1x+jzIiB7D7GMVtU8Vc3Lzc2NtYtzzpXpzDPhk0+ga9eAPqBdO2jePO2rmeKerK+Iqq4AWpez2wys+ggRaQh0Bn5ev1pE6gGnquqkEu/dIyInR38+HNha0diccy4eInDYYfZzII3VIja3x7RpSZ53PLHi7eZ6dPTRTkQGAGXOaKKq04HCaNvDROA2IE9ErojuMgQYWeqw+4FbRGQmcB9wffxfwznnKu7GG6FTp4BGV0cisG6dTb2RpuJtnjkba5xW4DvggvIOUNUbY7xd9Jt6WVUXl9r/B+D0OONxzrkqO+wwuOceqwnq3TvBJ+/e3VrAp0yBk9Nz+jrROFKniLQGmqnqOyLSEfhWVb8LOLZy5eXlaX4GDEZxzoWjoAAOPtiaC95+O4BpOLp2tYn7PvggwSeuGhGZr6p55e0XbxvEA8CK6M8rgaDnRHTOucBlZcENN8A779hqdAnXu7ctILQ8PXvsx5sgcotWklPVr4H9AovIOeeS6JJLrARx330BnLyou+vU9FxjLd4Esb2SxznnXEqrXRuefdbWjUi4tm2hVau07e4a74V+pohcLuYqYE6QQTnnXDJ17mwLwiWciJUiXnstwMmfghNvghiGzcf0PFAH67bqnHMZY+lS62z0/vsJPnEkAps2BThsOzjxjqRW4P9U9VzgAVVN/4nOnXOuhNxca08ePjzBJ+7aFbKz07KaKd6BclcDZ0Zf9hGRy4MLyTnnkm+ffWDwYHjxRZuGI2Hq1YMuXTI3QQC/VNUJAKr6IvDr4EJyzrlwDBkCderA3Xcn+MSRCCxZAl98keATByveBFG6dcWrmJxzGadxYxg0CMaNgy+/TOCJ07S7a7wJYqOINAIQkSbsnjCccy4jXHcd/OUvCV7G4aCDbMh2mlUzxTsX0y3AOBF5E1sM6NrgQnLOufDst59VNSVc797w6KOweTPUrRvAByRevCWIb4BLgFeA/kB6VaQ551wFPfkk3HtvAk8YicDWrTBrVgJPGqx4E8QTwEJs/MP7wD8Ci8g551LAzJlwxx3wQ+mlziqrc2crOUyenKATBi/eBPEDcJKqnoMt/vN9cCE551z4brzRbvgfeCBBJ8zOtinAp0wJaAGKxIs3QRyiqksBVHUJcGRwITnnXPgOPRTOPx/+7/9sxu6EiETg66/h008TdMJgxZsgaopIDQARqUn8jdvOOZe2br4ZNm6EUaMSdMJevew5TXozxZsgHgPGishJwOjowznnMtrRR8P110P79gk6YcuWcOSRaZMg4ioJqOpTIrIUOBF4TFXnBRuWc86lhoT2ZAKrZnrgAVi/Hho0SPDJEyvudR1UdZ6q/tWTg3OuuvnpJ1u7esuWBJwsErG1Tl9/PQEnC5Yv/OOcc+X48EPr1fT3vyfgZJ06QcOGaVHN5AnCOefK0bkznHSSlSK2l15fs6KysqBHj7To7hpYghCR4SIyR0Rmi0inUtteEpFZ0ccyEbmk1PauItI7qNicc66ibr0Vli+Hf/0rASeLROD7761/wSLbAAASf0lEQVRoksICSRAi0g2oq6qdgbOAYSKSVbRdVfupahdsXqf3gSdLHNsGG6mdhXPOpYgePaBDB5sKfMeOKp7s9NPtOcVHVQdVgugGjAVQ1XXAbOCoGPsNAMap6k4AEdkbuBt4KKC4nHOuUkSsFHHwwQkYONesmWWbFG+HCCpB5ACrSrxeCTQpuYOIZAN9gPHR1wLcB9wI/LSnE4vIQBHJF5H81atXJzpu55zbo759bUmHJk3K37dckQjMnZvAYdqJF1SCWAPklnjdDJvPqaRBwJjoetcAlwGdgMexJDFMRA4qfWJVHa2qeaqal5ubW3qzc84F7uuvYf78Kp4kEoHCQvjPfxIRUiCCShAzsOojRKQhNsHfoqKNIlIPOFVVJxW9p6pjVPWYaNvECOBWVf08oPicc65SVO3aPnBgFTshHXcc5OSkdDVTIAlCVacDhSIyG5iITROeJyJXRHcZAowM4rOdcy5IIvDHP8KCBfDqq1U4UY0a1lg9daqVJFKQaAj9cEXkCFVdXNXz5OXlaX5+fiJCcs65uG3fbquItmwJb7xhSaNSxo2D/v1h3jw4/viExlgWEZmvqnnl7RfKQLlEJAfnnAtLrVowdCi89RbMmVOFE/XsCXvtlbLVTD6S2jnnKuHSS+GAA2DhwiqcJCcHOnZM2QTh6zo451wl1KkDn30GtWtX8USRiA2wWLUKmjZNSGyJ4iUI55yrpKLk8NlnVThJJGLPVWrxDoYnCOecq4Inn7TlSSs9rVK7dtC8eUpWM3mCcM65KjjzTKhfH4YPr+QJRGwp0mnTEjDJU2J5gnDOuSrYd1+48koYPx6WLKnkSSIRWLfOpt5IIZ4gnHOuiv7wB2uPGDGikifo3h1q1ky5aiZPEM45V0VNmtjUG6+8Aps2VeIEDRvaikSeIJxzLvP86U/w+edQr14lTxCJWEv38uUJjasqPEE451wCNGpkBYHCQtiypRIn6B1dRHPq1ITGVRWeIJxzLkG2b7d1gG69tRIHt20LrVql1CpzniCccy5BatWCo46CRx+FH3+s4MEiVs00fTps2xZIfBXlCcI55xLoppusiunBBytxcCRirdxvvJHwuCrDE4RzziVQ27ZwzjkwahT8tMfFk/ega1fIzk6Z3kyeIJxzLsFuuQXWr4d//rOCB9arB126eIJwzrlM1a6dLTV91VWVODgSsSHZX3yR8LgqyhOEc84F4LTTICurEutWF83umgLdXT1BOOdcQF58EfLyYOvWChx00EFw8MEpUc3kCcI55wKy776wYEEl2iIiEZg5EzZvDiSueHmCcM65gHTtCiecYJP4FRRU4MDeva3YMXNmYLHFwxOEc84FRMRGVS9bBk89VYEDO3eGunVDr2YKLEGIyHARmSMis0WkU6ltL4nIrOhjmYhcIiJNRGSKiLwVfb9tULE551yy9OoF7dvD3XfDzp1xHpSdbVOAT5lSiVbuxKkZxElFpBtQV1U7i0hDYIKI9FTVAgBV7RfdrwbwEvAkcBRwg6ouEpEIcBbwSRDxOedcsojAAw9YFdNeFbklj0Rg4kT49FMbfReCQBIE0A0YC6Cq60RkNpYAFpTabwAwTlV3AgsBROQ54DSgfUCxOedcUp1ySiUO6tXLnqdMCS1BBFXFlAOsKvF6JdCk5A4ikg30AcaXfF9Vfwn0Ah6KdWIRGSgi+SKSv3r16oQG7ZxzQdm4Ea67rgLNCi1bwpFHhtoOEVSCWAPklnjdDPih1D6DgDGqVsEmIu2KNqjqvD3FpqqjVTVPVfNyc3Nj7eKccymndm14+WW4444KNCtEIjZx3/r1QYa2R0EliBlY9RHRNojOwKKijSJSDzhVVSeVOOaXItI9ur0lsCOg2JxzLulq1oQbb4T33oPXXovzoEjEGi9efz3Q2PYkkAShqtOBwmjbw0TgNiBPRK6I7jIEGFnqsLuBK0RkBvAocH0QsTnnXFguughatIC77orzgE6doEGD0KqZgmqkRlVvjPH23Ojzy6q6uNT+64F+QcXjnHNhy86GoUPhmmus5ujkk8s5ICsLevQo7u4qkpQ4i4QyUK50cnDOuerisstg4EBo1izOA3r3hu+/hw8+CDSuWHwktXPOJVHduvDYYzYfX1xOP92eQ6hm8gThnHMhWLQI/vrXOHZs1gw6dPAE4Zxz1cULL9i4iI8+imPnSATmzoW1awOPqyRPEM45F4Krr4a994bhw+PYORKBwkJbpi6JPEE451wIcnLg97+H556DpUvL2fm44+yAJFczBdbNNSwFBQUsX76crRVawim91K5dm/3335+srKywQ3HOVcG118KoUbZexN//XsaONWpYY/XUqVaSqNCsf5WXcQli+fLl1K9fn9atWyNJ7jOcDKrKmjVrWL58OW3atAk7HOdcFTRrBkOGxDm8IRKBp5+G/Hw4/vjAY4MMTBBbt27N2OQAICLk5OTgExU6lxnuvjvOHXv2tEwyZUrSEkRGtkFkanIokunfz7nqRhWmTYOVK8vYKScHOnaEyZOTFldGJgjnnEsny5ZZDdL995ezY+/eVsW0alU5OyaGJ4gE69+/P+3ateOggw6iS5cubNmypdxj3n33XcaMGZOE6JxzqahVK7jgAnjkEVizpowdIxF7fvXVpMTlCSLBnn76aR588EGuuuoqZs2aRZ06dco95vjjj+fyyy9PQnTOuVR1882waROMLD3PdUnt2kHz5knr7ppxjdSpZNasWTz00ENs2LCBF198kc8++4ybb76Z7du307BhQ9q2bcsJJ5zAPvvsw8KFC+nbty933nknBQUFfP311zRo0IBnn32W+vXrh/1VnHMBO+IIOPtsSxDXXWezfO9GxJYiffFF2LHDFpkIUGYniCFDYOHCxJ6zXTt48MG4d9+0aRPTp09n48aNDB06lOeff57GjRuzYcMGIpEIJ5xwwi77T58+nXnz5tG8eXPGjh3Lc889x2WXXZbY7+CcS0m33ALz5sGSJTY2Lqbzz7dxEevXQ6NGgcaT2QkiBXTt2hWAJUuW0LVrVxo3bgxA/fr1Oe+883bbv1evXjRv3hyAY489lldeeSV5wTrnQtWhA3z9tS0DsUc9e9ojCTI7QVTgTj8oRW0Qhx12GEOHDmXNmjXk5OSwfv16xo8fzx//+MeY+wPstddeFBYWJjVe51y4srJsldGlS+Hww8ONJbMTRAqpV68eI0aM4IILLqCgoICcnBxOOeWUsMNyzqWgiy6COXPgyy9tFbqwiKqG9+lVlJeXp/n5+bu898knn9C2bduQIkqe6vI9nauOXnvNVhp97DFbfS7RRGS+quaVt593c3XOuRTTvbvNpnH33VbdFBZPEM45l2JE4NZbrcH6mWfCi8MThHPOpaAzzoCjj4aJE8OLIbBGahEZDpwEKHCTqr5dYttLQFEH3gOB24FxwGNAK6AGcJ2qvhdUfM45l8pEbEaNpk3DiyGQBCEi3YC6qtpZRBoCE0Skp6oWAKhqv+h+NYCXgCeBwcBrqvq0iOQCzwDdg4jPOefSQXRIFOvW2cjqZE/kHFQVUzdgLICqrgNmA0fF2G8AME5VdwJvAy9Ej1kNrAsoNuecSxsLF0LLlklfbRQILkHkACXno10JNCm5g4hkA32A8QCq+q6qbotuuxIrQexGRAaKSL6I5KfiojmVmc21SN++fQOMzDmXjo44wmbUGDbM1o1IpqASxBogt8TrZsAPpfYZBIzRUgMxRORXQG1VfSHWiVV1tKrmqWpebm5urF1CVZnZXJ1zbk+ysuCGG+Cdd2DmzOR+dlAJYgZWfUS0DaIzsKhoo4jUA05V1UklDxKRXwIHqGp5y2akjfz8fC6//HJ69OhBv379GD9+PACPPvooxx13HL/85S8pKCjgjDPO4M0332Tw4MEhR+ycSzUDBlh7xLBhyf3cQBqpVXW6iHQXkdnRt24C8kSkvar+DRgC7DLruYicB4wBFohIr+jbvVQ1/jqaGLp02f2988+HK66AzZuL198oacAAe/z4I5x77q7bZs2K/7M3btzI7373u5+n6167di1//vOfOe2003jjjTd45ZVXaNCgAVlZWbzyyiv07duXUaNGxf8BzrlqoXZtuP56uPZaWLzYqp2SIbBurqp6Y4y350afX1bVxaX2fx54Pqh4wrBkyRJOO+00RowY8fN7BQUFZGVl8fDDDzN69GgOOOAALrzwwhCjdM6lg4ED4aSTkpccIKTJ+konhyCVdcdft27Z2xs3rliJobRDDjmESZMmMXjwYFq0aMG3337L1VdfzYQJEwAYOnQoAwcOpFOnTrRq1aryH+Scy3j16hWvEaGanC6vPptrgOrXr89jjz1G//79AcjOzuaRRx4B4O6772bu3Lnk5OTQokULAH744Qcef/xxBgwYEFbIzrkUN2QIrFqVnCk4fDbXNFVdvqdzbldjx0KTJnDWWZU/R7yzuXoJwjnn0kgyVyD2yfqcc87FlJEJIp2rzeKR6d/POZcaMi5B1K5dmzVr1mTsRVRVWbNmDbVr1w47FOdchsu4Noj999+f5cuXk4rzNCVK7dq12X///cMOwzmX4TIuQWRlZdGmTZuww3DOubSXcVVMzjnnEsMThHPOuZg8QTjnnIsprUdSi8hq4JtKHt4Y+DGB4YQpU75LpnwP8O+SqjLlu1T1e7RS1XIX1EnrBFEVIpIfz1DzdJAp3yVTvgf4d0lVmfJdkvU9vIrJOedcTJ4gnHPOxVSdE8TosANIoEz5LpnyPcC/S6rKlO+SlO9RbdsgnHPOla06lyCcc86VwROEc865mKplghCRgSIyT0TeEZE/hx1PVYjIaSIyW0TeEJFRIslYqTbxRKRB9Hs8GHYsVSUiw0VkTvT7dAo7nsoSkT4i8rWItAs7lqoSkftFZJaIvC0ivcKOp7JEpK6IPBv9Hm+JyElBfl61TBDARuAE4CSgo4jUCDmeSonGfQvQU1VPBtYCfcKNqnJUdT1wcdhxVJWIdAPqqmpn4CxgmIhkhRxWpajqv4HHw46jqkSkL7BWVbsA3YFbRSRdr33NgJGq2gm4HOgf5Iel6y+pSlR1HJYgVgKLVXVnyCFVVhZwvapujb5eDGSHGI+DbsBYAFVdB8wGjgo1IrcUeBRAVTcDy4G0LGmr6peq+raI/AXIB/4R5OdVywQBoKpvAQcA+4rIyWHHUxmqulVV3wMQkUZAX2BCuFFVeznAqhKvVwJNQorFAaq6WFXXAIjIWcDcNL4pBEBV/wgcA9wbZGmo2iQIEfnfaB3kQhFpBaCqW4AxQMdwo6uYEt9lVvR1A2AUcJ2qbg81uAoo/T0yxBqg5Bw3zYAfQorFlSAiXYGTVTVt27lEpK2I1AJQ1aXAF9hNSSAybsGgPVHV2wFEZB/gGRE5I3oXcTYwMdTgKqjouwCISH3gb8CNqroivKgqruT3yCAzgAHAUBFpCHQGhocakUNEOgNnAteFHUsVdQZOBMZGr2WtVDWw5TOrTYIooqo/icjjwBwR2QFMVdXZIYdVKSKyNzAVaAT8K9qB6TlVfSTUwKoxVZ0uIt1FpOj/1E2qWhBqUNVctAp5ErAQmBn9O7lEVb8KNbDK+TswRkT6A4XADUF+mI+kds45F1O1aYNwzjlXMZ4gnHPOxeQJwjnnXEyeIJxzzsXkCcI551xMniCcC0h0srtGcex3oogcmoyYnKsITxDO7YGIDIhO9FZZv1XVtdFzNReRx0XkXRGZKSKvi8iZ0f3+C1xS5YCdS7BqN1DOuWQQkV9gk8QVjd5/GRvtPjP6Xn3gbhF5T1U/FpEjwovWudg8QThXDhH5DTa18k5sXqWrVXWViDQGHgT2j257GfiFqg4BjgbmR0/xW+CvRckBQFU3AFeV+JgtIlIj3SeRc5nFq5icK9uhwGlAN1XtCtwH/F9028PAqOg6A6cBB5Y4rinwbYlzvF3O53wLNE5QzM4lhCcI58pWAIwtmk9JVfOBWtFttVV1XvT9QuCfJY5rBBSt0/E50KGcz9lKgLNyOlcZniCcK1tt4FdFqw6KyDFY0gAoEJHDo+8L8GugaHKzDViSAJtg7caSy49Gl44cISJNo281AtYH+k2cqyBvg3CubB8D3wMzRKQQ+BEYHN12DTAq2pV1O/A6sG9022qiCwWp6loR6QMMF5G7Spx7pKoWLS6UGz23cynDZ3N1rpJE5Frgn6r63+jr64AvVXWCiBwGXKmqg8s8SfG5pqlqzwDDda7CvAThXOW9AjwsIpuwdomlqjoBQFU/FZG41qKOdnFdElyYzlWOlyCcc87F5I3UzjnnYvIE4ZxzLiZPEM4552LyBOGccy4mTxDOOedi+n9uuzbt+I8lOAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot误差曲线\n",
    "train_means = grid.cv_results_['mean_train_score']\n",
    "test_means = grid.cv_results_['mean_test_score']\n",
    "\n",
    "Cs = np.logspace(-3, 3, 7)\n",
    "x_axis = np.log10(Cs)\n",
    "\n",
    "plt.plot(x_axis, np.array(train_means), 'r-', label='Train')    \n",
    "plt.plot(x_axis, np.array(test_means), 'b--', label='Test')    \n",
    "\n",
    "plt.legend()\n",
    "plt.xlabel( 'log(C)' )                                                                                                      \n",
    "plt.ylabel( 'accuracy' )\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最佳正则参数{'C': 1}，当C越大时，模型预测效果变好，准确率升高。但是当C过于大时，模型性能开始陡然下降，这是因为正则参数越大，代表模型可以容忍分错的松弛变量范围越窄，此时要求模型把更多的样本分类正确，模型复杂度升高，容易过拟合，造成在测试集上的预测效果不好。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1、RBF核的SVM\n",
    "\n",
    "#### （1）网格搜索找最佳参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:244: ConvergenceWarning: Solver terminated early (max_iter=10000).  Consider pre-processing your data with StandardScaler or MinMaxScaler.\n",
      "  % self.max_iter, ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:244: ConvergenceWarning: Solver terminated early (max_iter=10000).  Consider pre-processing your data with StandardScaler or MinMaxScaler.\n",
      "  % self.max_iter, ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:244: ConvergenceWarning: Solver terminated early (max_iter=10000).  Consider pre-processing your data with StandardScaler or MinMaxScaler.\n",
      "  % self.max_iter, ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:244: ConvergenceWarning: Solver terminated early (max_iter=10000).  Consider pre-processing your data with StandardScaler or MinMaxScaler.\n",
      "  % self.max_iter, ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:244: ConvergenceWarning: Solver terminated early (max_iter=10000).  Consider pre-processing your data with StandardScaler or MinMaxScaler.\n",
      "  % self.max_iter, ConvergenceWarning)\n",
      "/Users/apple/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:244: ConvergenceWarning: Solver terminated early (max_iter=10000).  Consider pre-processing your data with StandardScaler or MinMaxScaler.\n",
      "  % self.max_iter, ConvergenceWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise-deprecating',\n",
       "       estimator=SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma='auto_deprecated',\n",
       "  kernel='rbf', max_iter=10000, probability=True, random_state=None,\n",
       "  shrinking=True, tol=0.001, verbose=False),\n",
       "       fit_params=None, iid='warn', n_jobs=None,\n",
       "       param_grid={'C': [0.001, 0.01, 0.1, 1, 10, 100, 1000]},\n",
       "       pre_dispatch='2*n_jobs', refit='accuracy', return_train_score=True,\n",
       "       scoring={'accuracy': make_scorer(accuracy_score)}, verbose=0)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 线性SVM\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.metrics import make_scorer, accuracy_score\n",
    "\n",
    "\n",
    "# 需要调优的参数\n",
    "tuned_parameters = { 'C': [0.001, 0.01, 0.1, 1, 10, 100, 1000] }\n",
    "\n",
    "scorings = {'accuracy': make_scorer(accuracy_score)}\n",
    "\n",
    "# 采用5折交叉验证\n",
    "svc_rbf =  SVC( kernel='rbf', max_iter=10000, probability=True)\n",
    "grid = GridSearchCV( svc_rbf, tuned_parameters, cv=5, scoring=scorings, refit='accuracy', return_train_score=True)  \n",
    "\n",
    "grid.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7552083333333334\n",
      "{'C': 0.1}\n"
     ]
    }
   ],
   "source": [
    "print(grid.best_score_)\n",
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'mean_fit_time': array([0.07013097, 0.07396665, 0.0789794 , 0.06414919, 0.08768091,\n",
       "        0.17671156, 0.23615174]),\n",
       " 'std_fit_time': array([0.00398802, 0.00390811, 0.01457414, 0.0014919 , 0.00207958,\n",
       "        0.00434182, 0.0328415 ]),\n",
       " 'mean_score_time': array([0.00419993, 0.00460911, 0.00459342, 0.00371552, 0.00380578,\n",
       "        0.00330811, 0.00349579]),\n",
       " 'std_score_time': array([5.12060503e-05, 2.99832194e-04, 5.40760404e-04, 2.14072695e-04,\n",
       "        4.45685400e-04, 1.40273335e-04, 5.09235041e-04]),\n",
       " 'param_C': masked_array(data=[0.001, 0.01, 0.1, 1, 10, 100, 1000],\n",
       "              mask=[False, False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'params': [{'C': 0.001},\n",
       "  {'C': 0.01},\n",
       "  {'C': 0.1},\n",
       "  {'C': 1},\n",
       "  {'C': 10},\n",
       "  {'C': 100},\n",
       "  {'C': 1000}],\n",
       " 'split0_test_accuracy': array([0.64935065, 0.64935065, 0.72727273, 0.74675325, 0.70779221,\n",
       "        0.69480519, 0.62987013]),\n",
       " 'split1_test_accuracy': array([0.64935065, 0.64935065, 0.75974026, 0.71428571, 0.71428571,\n",
       "        0.7012987 , 0.69480519]),\n",
       " 'split2_test_accuracy': array([0.64935065, 0.64935065, 0.75974026, 0.73376623, 0.73376623,\n",
       "        0.67532468, 0.66883117]),\n",
       " 'split3_test_accuracy': array([0.65359477, 0.65359477, 0.77124183, 0.79738562, 0.80392157,\n",
       "        0.77124183, 0.66013072]),\n",
       " 'split4_test_accuracy': array([0.65359477, 0.65359477, 0.75816993, 0.76470588, 0.78431373,\n",
       "        0.7124183 , 0.65359477]),\n",
       " 'mean_test_accuracy': array([0.65104167, 0.65104167, 0.75520833, 0.75130208, 0.74869792,\n",
       "        0.7109375 , 0.66145833]),\n",
       " 'std_test_accuracy': array([0.00207782, 0.00207782, 0.01475412, 0.02827433, 0.03844405,\n",
       "        0.03240446, 0.02113373]),\n",
       " 'rank_test_accuracy': array([6, 6, 1, 2, 3, 4, 5], dtype=int32),\n",
       " 'split0_train_accuracy': array([0.6514658 , 0.6514658 , 0.77687296, 0.83550489, 0.88599349,\n",
       "        0.96254072, 0.99674267]),\n",
       " 'split1_train_accuracy': array([0.6514658 , 0.6514658 , 0.76221498, 0.83061889, 0.89739414,\n",
       "        0.95439739, 0.99837134]),\n",
       " 'split2_train_accuracy': array([0.6514658 , 0.6514658 , 0.76710098, 0.83224756, 0.8990228 ,\n",
       "        0.96416938, 0.99837134]),\n",
       " 'split3_train_accuracy': array([0.6504065 , 0.6504065 , 0.75447154, 0.81138211, 0.88130081,\n",
       "        0.95772358, 0.99349593]),\n",
       " 'split4_train_accuracy': array([0.6504065 , 0.6504065 , 0.76585366, 0.82601626, 0.87642276,\n",
       "        0.96422764, 0.99512195]),\n",
       " 'mean_train_accuracy': array([0.65104208, 0.65104208, 0.76530283, 0.82715394, 0.8880268 ,\n",
       "        0.96061174, 0.99642065]),\n",
       " 'std_train_accuracy': array([0.00051895, 0.00051895, 0.00726989, 0.00845853, 0.00886212,\n",
       "        0.00390927, 0.00189497])}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.cv_results_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （3）评价指标曲线图\n",
    "\n",
    "- 不同正则参数C下，对应的**正确率**曲线"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEICAYAAABF82P+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xmc1XP///HHK6WM9k0I2WUNIwqFZE2LSGQpSygRF1muL66LSpd9ydJGtqikUq7wS+tFxeQqkey6hLYpNK3TzOv3x/tMTTWaUzNnPmfOPO+329zmfM7n8znn9WE6r/P+vN/v19vcHRERka2VizoAERFJTkoQIiJSICUIEREpkBKEiIgUSAlCREQKpAQhIiIFUoIQEZECKUGIiEiBlCBERKRA5aMOoChq167tDRo0iDoMEZFSZfbs2cvdvU5hx5XqBNGgQQMyMjKiDkNEpFQxs4XxHKdbTCIiUiAlCBERKZAShIiIFChhfRBm1gZ4Cmjr7nO22tceuA3IAYa5+wtmVgF4FjgU2ADc5O5f7+j7Zmdns2jRItatW1fka0hWlSpVon79+lSoUCHqUEQkhSUsQbj7WDM7duvnzawa0A04g5AI3jCzqUBTYL67dzWz+sBA4Lwdfd9FixZRpUoVGjRogJkV7SKSkLuTmZnJokWL2H///aMOR0RSWBS3mE4ERrv7eg+rFT0PnEtIGIMB3H0RsMTMau7oi69bt45atWqlZHIAMDNq1aqV0i0kEUkOUSSIWsCSfNuLgbpAmrtnFfD8Fsysq5llmFnGsmXLCnyDVE0OeVL9+kSkEH/x2VfcokgQmUD+CRr1gKXAGjNLK+D5Lbj7QHdPd/f0OnUKnechIpIacnJg3Dho1Qr22gt++inhbxlFgpgFtDGzCha+Ct8A/BuYDHQGMLO9gT3cfUUE8RVJp06daNSoEQcddBCnnXYaa9euLfScTz75hEGDBpVAdCJS6vzyCzzwADRoAK1bw2efwV13QVpaoacWVYnNpDaztsAu7j7KzIYQEoIDr7v7AjP7AXjWzKYA2UCPkoqtOL3++utMmTKFOXPm0LNnz7jOady4MY0bN05wZCJSauTmwgcfwIABodWQkwNnnQVPPQUXXAAlNIIxoQnC3f+R7/GYfI9HACO2OnYDcF2xBtCzJ8yZU/hxO6JRI3jyybgOnTJlCk899RSrVq1i1KhRfPPNN9xzzz1s2LCBatWq0bBhQ5o0aUL16tWZM2cObdu25YEHHiA7O5uffvqJqlWr8uabb1KlSpXivQYRSU5LlsCLL8KgQfDjj1CnDtx+O1x3HRx4YImHU6prMZUGq1evZuLEiWRlZdGrVy9GjhxJ7dq1WbVqFeeddx5NmjTZ4viJEycya9Ys9txzTwYPHszw4cO59tprI4peRBIuNxcmTw6thdGjYeNGOP10eOghaNsWKlaMLLTUThBxftNPpNNPPx2Ar7/+mtNPP53atWsDUKVKFS6++OJtjj/33HPZc889ATjuuOMYP358yQUrIiVn+XIYOhQGDoRvv4WaNeHmm6FrVzj00KijA1RqI+F22203AA477DCmTp1KZmYmAH/++ScjRoz4y+MBypUrR25ubskEKiKJ5w7Tp0OnTrD33nDHHVC3Lrz6auiMfuyxpEkOkOotiCSy++67069fPzp27Eh2dja1atWiefPmUYclIiVh5Up45ZVwG+mrr6BaNbj++tBaOPLIqKP7SxYmM5dO6enpvvV6EF999RUNGzaMKKKSU1auU6TUcodZs+CFF2D4cFi3Dho3hhtugEsuKZFhqn/FzGa7e3phx6kFISJSnP78E15/PSSGzz+HypXhqqtCi+HYbcrTJTUlCBGR4jB7driFNGwYrF4dksGAAXDppVBKh6orQYiI7KysLHjzzdBamD0bdtstJITrr4cTToBSXjdNCUJEZEd9/nloHbz6KqxaFTqan3kGLr8cqlePOrpiowQhIhKPtWthxIiQGGbMCBPYOnQInc5NmpT61kJBlCBERLbnq69CUnj5Zfj99zBP4fHH4coroVatqKNLKE2UK2Y7U801T9u2bRMYmYjEbf360NncvDkcfjg89xycc04oifHVV3DrrSmfHEAtiGK3M9VcRSRJfPttKH0xdGgohXHggfCvf0HnzmHGcxmT8gnitNO2fa5DB+jWDdasgfMKWPW6c+fws3w5XHTRlvumTNmx98/IyGDAgAEsXLiQypUr07FjRzp06MALL7zAkCFDOOCAA3jttddo164dM2fOpEePHjzzzDM79iYisvOys2Hs2DAS6cMPYZddQpG866+HFi2gXNm90ZLyCSJKWVlZXH/99ZvKda9YsYIHH3yQli1bMn36dMaPH0/VqlWpUKEC48ePp23btkoOIiXlxx9DWe0XXwxltvfdF3r3hquvhljBzLIu5RPE9r7xp6Vtf3/t2jveYsjv66+/pmXLlvTr12/Tc9nZ2VSoUIFnn32WgQMHss8++3DppZfu/JuISPw2boR33w2thfffDyOPWrUKrYWzzw6tB9mk7LadSsAhhxzCuHHj+OWXXwD4+eef6dChw6b9vXr1YvLkySxcuDCqEEXKhg0boE+fsGxn27ZhHsN994V1nceODfealRy2kfItiChVqVKFAQMG0KlTJwAqVqzI888/D8BDDz3EjBkzqFWrFnvvvTcAS5cuZejQoXTu3DmqkEVSz7JloTNx2rTQSnj2WTj/fCivj7/CqJprKVVWrlOkSD7/HFq3hsWLYciQsA6DxF3NVbeYRCQ1vf02NG0aRinlLdIjOyRhCcLM+prZNDObamZNt9rXPPb8DDPrnu/5883sSzObYmYjExWbiKSw3Fz45z+hfftQIykjIxTOkx2WkJtwZtYCSHP3ZmZWDRhtZme7e7aZGXAfcC6wDnjLzCa4+w/ACcCl7v55Ud7f3bEUrIuSpzTfFhRJqKysMIlp1KhQCmPAAKhUKeqoSq1EtSBaAIMB3P0PYCpwVGxfbeB7d1/j7rnAh0CD2L504FEzm2BmR+zMG1eqVInMzMyU/RB1dzIzM6mkP3qRLf30E5x8MoweHdZ2HjpUyaGIEtWNXwtYkm97MZA3T305cLiZ1QVWAa2BN2L7Orv7cjOrD7wEtNz6hc2sK9AVYN99993mjevXr8+iRYtYtmxZMV1K8qlUqRL169ePOgyR5DFtWrillJ0d5jmcc07UEaWERCWITKAOkPcpXQ/4FMDd3cx6ASMISWOcu6+I7Vse+73IzFaaWQV3z87/wu4+EBgIYRTT1m9coUIF9t9//8RclYgknwED4KabQt2ksWNDtVUpFom6xTQJ6AwQ64NoBszL2+nuHwNnAd8DD8SO62Zml8ce1wOqb50cREQ2yc6G7t3DegxnngkzZyo5FLOEJAh3nwjkmtlU4B3gXiDdzLrlO+x+oL+7r45tvwS0NbMpwOvA3xIRm4ikgOXL4ayzQhnu22+H8eNTaiW3ZJGwqYTuflcBT8+ATS2EOu7+fr7j1wIXFXCOiMhm8+aFyW+//QavvAJXXBF1RCkrkoly7r7Y3btG8d4iUoqNGROW91y/PnRMKzkklGZSi0jyc4cHH4R27cIKbxkZ0Lhx1FGlPFWrEpHktno1dOkCI0fC5ZeHFd922y3qqMoEJQgRSV4LF4by3HPnwsMPhw7pFK6SkGyUIEQkOf3nP3DhhaG/Yfz4gtcHloRSH4SIJJ9Bg+CMM8LQ1VmzlBwiogQhIskjOxt69ICuXUOCmDULDjss6qjKLCUIEUkOmZmhhlL//nDbbeG2Uo0aUUdVpqkPQkSi98UX0KYNLFoUqrBedVXUEQlKECIStXfeCau9Va4MU6fCSSdFHZHE6BaTiETDHfr0CcNYDz0UPv1UySHJqAUhIiVvzRq4+moYPhwuuwwGD9bktySkBCEiJevnn0N/w5w50K8f9OqlyW9JSglCRErORx+FyW9r14a+h1atoo5ItkN9ECJSMl58EU4/HapWDfMblBySnhKEiCTWxo1wyy1wzTXQvHlIDg0bRh2VxEEJQkQSZ8UKOPdcePpp6NkTJkyAmjWjjkripD4IEUmM+fPDym8//xxuL3XpEnVEsoOUIESk+I0bFya/paXB5MnQtGnUEclO0C0mESk+7mHoaps2cPDBYfKbkkOppRaEiBSPNWvg2mvhjTegY0cYMiS0IKTUSlgLwsz6mtk0M5tqZk232tc89vwMM+sezzkiksQWLYJmzeDNN6FvXxg2TMkhBSSkBWFmLYA0d29mZtWA0WZ2trtnm5kB9wHnAuuAt8xsArD/X52TiBhFpJjMmAHt2oW1o8eOhQsuiDoiKSaJakG0AAYDuPsfwFTgqNi+2sD37r7G3XOBD4EGhZwjIsnopZfgtNNCJdaZM5UcUkyiEkQtYEm+7cVA3djj5cDhZlbXzHYDWgNzCjlnEzPramYZZpaxbNmyhAQvIoXYuDEs6nP11XDqqfDJJ3DEEVFHJcUsUQkiE6iTb7sesBTA3R3oBYwAZgNz3H3F9s7Jz90Hunu6u6fXqVNn690ikmgrV4Y1op94Am6+Gd57T5PfUlSiEsQkoDNArD+hGTAvb6e7fwycBXwPPBDPOSKSBL76Cho3hilTYNAgeOopKK/BkKkqIQnC3ScCuWY2FXgHuBdIN7Nu+Q67H+jv7qv/6hx1UIskkXffhRNPhD//hEmTwpBWSWkJS/3uflcBT88AMLN6QB13fz+Oc0QkSu7w8MNw993QqBGMGQP77ht1VFICImkbuvtioGsU7y0iO2Dt2tBSGDYMOnQIo5Y0v6HMUKkNESnYl1+GMhnDhkHv3mESnJJDmaIEISJbysmBxx6D44+HX34JK7/9/e9aFrQM0vADEdnsxx/hqqtg+vRQcG/gQKi7zXQkKSPUghCR0BE9eDAcfTTMmRP6GkaPVnIo49SCECnrFi8OHdHvvhvWjH7pJdhvv6ijkiSgFoRIWfbWW3DkkfDhh/DkkzBxopKDbKIEIVIWrVwZVny7+GLYf3/47DO45RYop48E2Ux/DSJlzQcfwFFHwfDh8I9/wMcfQ8OGUUclSUgJQqSsWL0auneHs8+GqlVDee7774cKFaKOTJKUEoRIWTBjRiiT8dxzcOutMHs2pKdHHZUkOSUIkVS2YQPccw+cckp4PGkSPP447LZb1JFJKaBhriKpat48uOIKmDsXunQJo5SqVo06KilF1IIQSTU5OaH6ano6/PZbWCf6xReVHGSHqQUhkkq+/z6UyvjoI2jXDgYMAK28KDtJLQiRVOAe6iYdc0y4tfTKKzBqlJKDFIlaECKl3a+/hlIZEyZAixahVMY++0QdlaSAuFoQZtY80YGIyE4YPjyUypgyBZ55JkyCU3KQYhLvLabGZjbGzC41M92WEonaihVw6aXQsSMcfDD8979w000qlSHFKq6/Jnd/BLgI2AV428xuM7PKCY1MRAr23nuh1fDWW/Dgg6FD+tBDo45KUlDcXzfcfSPwJ5AL1ABeNLOH/+p4M+trZtPMbKqZNd1q32VmNsPMppvZg/meP9/MvjSzKWY2cscvRySFZWXBDTfAuedCzZowaxb83/9BeXUlSmLE9ZdlZl2AdsC7wKXuvj72/FF/cXwLIM3dm5lZNWC0mZ3t7tmxQ64AzgN+B543s6Pd/XPghNjrf16kqxJJNR99BFdeGVZ8u/320HKoVCnqqCTFxfvVYxXQxt09/5PuPu8vjm8BDI4d84eZTQWOAj6L7X8M+BnIjMVwd+z5dKCpmeUAt7v7l/FeiEhKWr8e7rsPHnkkrNMwZQo0axZ1VFJGxJsgygMXAqPM7CIg293Hbuf4WsCSfNuLgboAZpZGSAgHxY5pDuTEjuvs7svNrD7wEtAy3gsRSTlz54ZSGfPmhWGsjz8OVapEHZWUIfH2QXQHxsQejwFuKeT4TCD/DJ16wNLY4yOBye6+ONYi+Q64E8Ddl8d+LwJWmtk2dYjNrKuZZZhZxrJly+IMX6QU2bgRHnoITjgBli6F8eNh0CAlBylx8SaIje6eA5s6q62Q4ycBnQFifRDNgLzbUd8Ths3mtV5aA2vMrJuZXR47px5QPV+fxSbuPtDd0909vY5miUqq+fbbcAvpnnugTRv44gs4//yoo5IyKt5bTBvMbDd3XxvP8FZ3n2hmZ8b6HiDcUko3s2Pd/TkzexmYbGYQ+iKuBRx41cyuJdxy6rnDVyNSWrnDCy+EDuhdd4XXXw/zHKyw72IiiRNvgugDvGZmo4GOQO/CTnD3uwp4ekZs3yhgVAH7L4ozHpHU8csvcM018P770LJlqLxav37UUYnElyDcfZqZfQ80Brq7+8LEhiVSBrjDG2+EZUA3bAirvd1wg1oNkjTinmHj7r8AoxMYi0jZsXw5dOsGI0dCkybw8suhZIZIEom3WN97ZvaLmU2M/Z6Q6MBEUta778JRR8GYMdC3L0ybpuQgSSneUUxzgPbufibQAZiVuJBEUtSqVdC1K7RqBbVrwyefwN13q1SGJK24q7m6+0wAd/8IODVxIYmkoGnTwmI+gwfDnXdCRgY0ahR1VCLbFW+C2HWr7W0msIlIAdatgzvugNNOC53P06ZBv35QsWLUkYkUKt4EMcbM+phZfTPrDWyvzIaIQFijIT0dHn003FqaOxdOOSXqqETiFu96EI8Cc4FewFx3fyyhUYmUZhs3Qu/e0LhxWNjn3/8Ok+AqawkVKV3iLff9oLvfC4xIcDwipdt334UCezNnhtXenn02rN0gUgrFe4uphpl1NLM9zKyqmVVNaFQipY07DBwYOqIXLIBhw8IkOCUHKcXiHV+3O3B27AdC3aSrExKRSGmzZEkoxz1+PLRoAUOHqlSGpIR4E8Q/8j12Cq/mKlI2vPNOSA5//glPPgk9ekC5uFfyFUlq8SaInoTEkAacC3wAXJeooESSXlYW3HprmNfQqBFMngxHHBF1VCLFKt5ifbfmPTazPYGHEhaRSLKbMQMuvzysD33nnfDAA6FEt0iK2eG2sLv/BjQo/lBEklx2Ntx7b5jLkJMDU6eGSW9KDpKi4h3menTsYTmgEbAqYRGJJKMFC0KrYfZsuOoqePppqKrBfJLa4u2DaEfog3DgF8KiQSKpzz2s03DHHZCWBm+9Be3bRx2VSImIN0G8DNRz95lmdhJQHViduLBEksCvv8LVV4eV3s45J6z0tueeUUclUmLi7YN4Avgt9ngx8GhiwhFJEqNGhTUbpk0Ls6H//W8lBylz4k0QdfKWGXX3n4C9EhaRSJT++CP0MVx0ERxwQCi4162blgGVMineBLFhJ88TKT3y1mx47bUwWunjj+HQQ6OOSiQy8X7QTzaz6yy4CZhW2Alm1tfMppnZVDNrutW+y8xshplNN7MH4zlHJGHWrw/zGU47Lazu9p//hLkNFbTsiZRt8SaI3oR6TCOB3YB7t3ewmbUA0ty9GdAa6G1m+f+1XQGcBzQD6pjZ0XGcI1L8vvwSTjwRHn44lMyYMweaNIk6KpGkEO96EA70d/eLgCfcPbeQU1oAg2Pn/gFMBY7Kt/8x4GfgJ+CC2OPCzhEpPrm5oXbS8ceH0Upjx4ZqrFqzQWSTuBKEmd1M+CAHaGNmhdVhqgUsybe9GKgbe6004G7gIMKM7E5AzvbO2SqWrmaWYWYZy5Ytiyd8kS0tWgRnnRVqKZ11FnzxBbRuHXVUIkkn3ltMl7j7aAB3HwVcXsjxmUCdfNv1gKWxx0cCk919caxl8h1wZyHnbOLuA9093d3T69Sps/Vuke17440wfHXmTBg0KLQc6m7zPUREiD9BrN9qu7BbTJOAzgBmVo3Q1zAvtu97oLGZ5U3Saw2sKeQckaJZuRIuuyz8HHZY6Gu49loNXxXZjnhnUmeZWU13X2Fmddk2YWzB3Sea2ZlmNjX21N1Aupkd6+7PmdnLhJFREPofrnX3NVuf4+7ZO3FNIluaNCnMbfjtN3jwQbjrrjBaSUS2K95/JX8HhpnZfwidybcVdoK731XA0zNi+0YBo+I8R2TnrFsH99wDTzwR5jPMmAEnnBB1VCKlRrwJYiHQBdgDeBHISlhEIsVh7lzo1CkMY+3ePQxjTUuLOiqRUiXePoiXgTmE+Q//JSQJkeSTkxOSwQknQGYmTJgA/fsrOYjshHhbEEuBU9z9WzM7FOiewJhEds5PP4W+hmnT4MILYcAAqF076qhESq14WxCHuPu3AO7+NWGoqkhycIdXXoGjjw7F9YYODes2KDmIFEm8CaK8me0CEBueqiEgkhwyM6FDh9ByaNQIPv88PNbwVZEiizdBDAAGm9kpwMDYj0i03n8/THobOxb+9S+YPBkaNIg6KpGUEVdLwN1fM7NvgZOBAe4+K7FhiWzHmjWh+mr//nDEEWExn0aNoo5KJOXEfasolhSUGCRas2fD5ZfDggXQsyc89BBUqhR1VCIpSQv/SOmwcSP06QMnnQSrVsHEiWECnJKDSMKos1mS3/ffw5VXhhXeOnaE556DGjWijkok5akFIcnLHYYMCf0LX34Jw4aFaqxKDiIlQglCktPSpdCuXai42rgxzJsHl14adVQiZYoShCSf8ePD8NX33oPHH4f/9/9gn32ijkqkzFGCkOSxejVcfz1ccAHUqwcZGWHVt3L6MxWJgjqpJTksXAgtW8J330GvXvDAA1CxYtRRiZRpShASvXXroH370O8weTI0bx51RCKCEoQkg5tvDhPgxo5VchBJIrq5K9F66SUYNAjuvhtat446GhHJRwlCojNnDnTrBi1ahLWiRSSpKEFINFauDP0OtWuHyW+77BJ1RCKyFfVBSMnLzQ2lM37+Oaz+VqcOkyfDFVfAhg1QuTLsvnu4+5SeDv/5T6iukfd83k+XLmE07A8/wPz5W+6vXBn22gvK6y9cZKcl7J+PmfUFTgEcuNvdP863722gZmzzAOB+d3/JzLoDVwOrgHnu3iNR8UmE+vULk+H694eTTuL330Njok6dMAUiKytMiahcORy+bBl8+ml4Lm9fbu7m6RLjx8Mtt2z7Nj/8APvvD488Ar17b04ceUlk/HioXh1GjQq1/7ZOQNdfHxo2CxaEGLZOQNWqlex/NpGSlpAEYWYtgDR3b2Zm1YDRZna2u2cDuPuFseN2Ad4GXomdejzQwt1/T0RckgQmToR774XLLgv9D4QP6cGD4bjjCl7vp1278JPHHdavhwoVwvall4Yir6tXb5lE6tYN+xs1Cq2N/PtWr4Zddw37588PSWL16rDURJ4bbgi/H3889KPnl5YWjge48UZ4990tk0f9+vDaa2H/00+H96hUKUztqFQJ9txz8+tPmBAWxsu/v1at8N8DwhQR9y33V6yo+YOSeObuxf+iofUwzN2/iG3fD4xz98+2Ou4aIMvdh8e2ZwOZQBZws7sv2t77pKene0ZGRrHHLwny88/hU2+PPWDWLNaX3525c0OppWSRmxuSxOrVIUyAb74JH9J5iWX1asjJCYkBQvKYMWNz8snKgqpVYdy4sP+KK+CDD0JSW7cu/D788FB/EODkk0Oh2vwaN4ZZsdVXjjkmrKSa3xlnwIcfbj72119D4shLHmeeGRbZA+jcObxnXnKpVCkk1I4dw/6nnw4tpfzJ55BDwvtC+O+RllYs/3klSZjZbHdPL+y4RN1iqgUsybe9GKib/wAzqwi0if3kOdvdl5vZUcBjwCVbv7CZdQW6Auy7777FHLYkzPr1cNFF4ffbb+Npu3PNFTByZPgA3m+/qAMMypULLYC821sQPiwPOeSvz7nuuvDzV159dctt97C8RZ5Ro0JSWbducwLJv8xFnz7hFldeglm3bsvSVC1bwuLFm89dt27L+L/9FpYv33L/unWbE8Rtt4WEl1+PHiFxrFwJBx4I558fpquccMJfX6eknkQliEygDrAstl0P+HSrY24ABnm+Joy7L4/9nhdLINtw901rYqenpxd/80cS47bb4JNPwqfhIYfw4APw+uuhbyBZkkNJMdt8ewxCP8r2tGq1/f19+mx//0cfbX//ihVbtm7WrQu3/QCys6FTJxg6NNwyO/HEkDwuvnjzLTpJXYm6izkJ6AwQ64NoBszL22lmuwNnuPu4fM/9y8xOjT0+HFiXoNikpL32WhiGdMcdcOGFvPEG3H9/GMh0zz1RBydVq4YBAvvsAwcdBEceGfpQIPTjPPMM/PLL5hbF5ZeHklkQWkOSuhLSBwFgZv2AJrHNuwmjmY519+fM7O/ATHf/MN/xdQmd1RWBNcAN7v7z9t5DfRClwLx54Wtn48YwcSILvitPo0bhqQ8+UD2+0iY3NzQETzopbF9xRbhd1qMHNGkSWkeS/OLtg0hYgtjum5od4e5fFvV1lCCS3B9/hIkMq1fDZ59BvXq4w5NPhtZDrVpRByhF4Q533QUDBoT/1ccdFxJFx45aKjzZxZsgIhkoVxzJQZKcexg+89NPMGIEv1eqx48/hm+Yt96q5JAKzMJIqUWL4PnnQ99Fly7w0ENRRybFRSOpJTEeeQTGjIFHHiH7xFPo0AFOOWXLeQaSGipXDnM6vvgiDL3t2jU8P2FC6MyeNk19FaWVEoQUvylTQnXWDh3wm2+hR4+wamjv3hpPn8rMwvyMvfcO20uWwKRJoYJ7o0ZhMqS+IJQuShBSvH75BS65JEwcGDyYJ540BgwI+aJLl6iDk5LUuXOYG5k3C/2660IrUkoPJQgpPhs2QIcOoVP67beZOKsKt98e5sf17h11cBKFtDS49tpQ2X3q1DC8GcJ8iyuvDLekdPspeSlBSPHp1SvUjHjxRWjYkCZNwiiXl19W3aCyzgyaNYM2sboJCxaEPoozzwzzLp5/Pswml+Sif7ZSPN58E556Cnr25LdTO7BqVShc17ev+h1kW8ccE24/DR0ahsR26xYm5/3wQ9SRSX5KEFJ08+eH+wgnn0zWfQ9z/vlw3nm6dSDbV6kSXHUVZGSEciDXXRfKs0Po0H7vvTAxT6Kj5VSkaFatCos5VK5Mzhsj6NS5AnPnhrUWNKtW4mEGTZuGHwhJ4ZFHQhHHQw6B7t1Dh3fVqpGGWSapBSE7zx2uuSaUCx0+nF5P7sU774SaPeeeG3VwUlqVKxcqtLz+OtSoERaD2ntvGDYs6sjKHiWLGDWyAAAP80lEQVQI2XlPPhnqdT/0EC9+35zHHw8lobt3jzowKe123TWsKTVzZqj91K4dNGwY9s2fH1qouv2UeEoQsnOmT99UnZXbb6dFC+jZM6y+JlKcTjgBXnkFjj02bD//fFhu9uCDw9/b71p/MmGUIGTHLV4c5jsccAC/9nmJXDf22w+eeCKsTCaSSI8/DsOHw157wd/+Fm4/3XVX1FGlJiUI2THZ2WGm9J9/snTQWE4+tyo33RR1UFKWVKgQvp9Mnx6KBHfsuPmLiXuYX7H1CnmyczSKSXbMPffAtGmsffEN2tzVkCVLVEJDonPssTBkyObtyZPDEOv99gtzK665RpWDi0ItCInfqFHw6KPkdruJLu93ZObMsFic1imWZNGsWfgz3X9/uPPOMPnummvCsqqy45QgJD5ffx2aCieeyAM1n2T48LAWwIUXRh2YyGbly4e/ycmT4fPPQ72nadOgSpWwf9myaOMrbXSLSQqXlRX+1VWsCCNHcubCXchaGwYxiSSro44Kq91t3BgSx5o1YdW7o48OE/EOPzzqCJOfWhCyfe5hBZgFC8gc8Bbssw+nnAKPPqqZ0lI6lC+/+XfPnqGsx9FHw403hjUr5K8pQcj2PfssvPEG3/Xsz2HXN+f556MOSGTn7LprGBb73XehA3vw4DCX4ttvo44seSlByF+bMQNuu42VZ3fk/PE34A4tW0YdlEjR1K4dysF8+SX06AEHHRSe/+ILzc7eWsIShJn1NbNpZjbVzJpute9tM5sS+/mfmXUxswpmNjB2/P8zs0MTFZvEYelSuPhiNtQ/gPZrXuGnn4zRozf/YxIp7Q45BPr0CbdKlyyBk06Cxo3DwkYSJCRBmFkLIM3dmwGtgd5mViFvv7tf6O6nAS2A/wKvAFcC8929OdAFeCIRsUkcNm6ESy/Fl2dy49EfMXl6BYYMgVNPjTowkcSoUwdeeCEkitNOC7Wfvvkm6qiil6gWRAtgMIC7/wFMBY4q4LjOwDB3zwHOyHfOImCJmdXc+gQz62pmGWaWsUxj1hLj3nth0iRswAs0aVWLf/wDLr886qBEEqdcufA3/s03oVUxcWIYBfXrr1FHFq1EDXOtBeQfH7AYqJv/ADOrCLSJ/UBocWQVcM4WU1zcfSAwECA9PV1L0hS3sWOhXz+yuvSg8lVXcW3U8YiUoN12C8UCrrkmlOzYa6/w/Hvvwemnh5HeZUmiWhCZQJ182/WApVsdcwMwyH3TumNrzCytkHMkkb77Dq68kk8bXkmDd57iww+jDkgkGnvsERYpAvj++1C+o2FDGDGibK2UmKgEMYlw+wgzqwY0A+bl7TSz3YEz3H1cvnMm5ztnb2APd9cE+ZKyZg20b8//bD9aLx9ClSrGUQXdFBQpYw48EN5/HypXDnUqTz45DPArCxKSINx9IpBrZlOBd4B7gXQz6xY7pCfw9FanvQIca2ZTgKHAbYmITQrgDjfeyJ+f/0SrGh+xZn153n0X6tYt/FSRsqBlS/jvf8PciR9/hDPPhJUro44q8cwjaC+Z2RHu/mVRXyc9Pd0zMjKKI6Sy7YUX2HjjTbQ+6Cs++PFgJkzQfAeRv5KVFVa5O+OM8N3queegUyeoXj3qyOJnZrPdPb2w4yKZKFccyUGKySefhEV/zz6HA88+iP79lRxEtqdy5ZAcAObODZPtDjwwTL7Lzo42tuKmmdRl2fLlcNFFrK+3H+WHvcIz/Y0bbog6KJHSo1GjsGhRo0bhe9YRR8CYManTka0EUVbl5ECnTvz7t2M5bOM8vlm+zZQTEYlDo0Zh3sT48aEgYLdusHZt1FEVDyWIsuqf/+TzD37jkl1GUr1uxU3jvUVkx5nB+eeHNSgmTYK0NNiwIRQH/N//oo5u5ylBlEXvvsviBwfSavfJVK1VgXHjwn1VESma8uXhsMPC44yMUAz50EPh73+HVauijW1nKEGUNT/+yJrLu9I6bSKZXpNx44z69aMOSiT1NG0aFmK88ELo2zcUunzhhXB3t7RQgihL1q6F9u3ZkFueascewLBhxnHHRR2USOrabz94/XWYNStUjx00qHQttKUEUZb06EHOf+dS/fVn+WB6Gm3aFH6KiBRd48Zhbez33w+FATMzoX37MEw2mSlBlBVDhvDSkBya1/+BlSe3KlXfYkRSgVlYrAjC4kSTJ8Oxx4bCgMlaNVYJoiz47DMm3ziCrjaItMP2VYe0SMSaNw9FAG+9FV59NSx9+s9/Jt+KdkoQqW7FCr5ufQftc4Zz8MEwYqRRoULhp4lIYtWoAY89Bl99FarFzp0bbj9B8ky0U4JIZbm5ZF7SjfN/GUD5arvz7nvlS1W9GJGy4MADYeRIePPNsP3116HPYuLEaOMCJYjU1qcPKydmUHHPmowZX4H99486IBH5K7vuGn4vWQLLloWaaK1awfz50cWkBJGi/L338fvu56ArmvL5/2rQtGnUEYlIPJo1gwUL4F//gunT4eijQ52nKG47KUGkooUL6XPhbK6t8RYb+7/ALuU1ZEmkNKlUCXr1Ch3ZN94YtvNGHm7YUHJxKEGkmvXrGX7GAO5dew8bTj2TXaqkFX6OiCSl2rXhmWegX7+w/eGHYcTTmDEl8/5KEClmxiVPctUP93FKw+UMHlFV8x1EUkDev+PKlUPSWFFCizGXL5m3kZLw06Nv0WZsF/auvprR02pTsWLUEYlIcTrxRPj005J7P7UgUsXcufxwz2B2q+i8O73aphmbIpJaypXbPF8i0SJtQZhZK6Cuu78YZRyl3u+/Q/v2nFFnLd/MgIr7qmEoIkWXsDxkZn3NbJqZTTWzbQZZmlkVoDvwer7nupvZbDObYmbPJCq2VOI5ufQ47j88/eMFMHIkFffdI+qQRCRFJOSrppm1ANLcvZmZVQNGm9nZ7p5/Se8Hgf9z9/X5njseaOHuvycirk2mT+ftsbuQk7tlD+4Be2Rx/AG/4w5vzdx2kYRD9lzFMQ3+IHujMebTvbfZf8Q+f3B4/VWs3VCO8bO3XaLtmP1+55C9svhzTXnen1tvm/3HH7CSA/ZYzYqsCnw4b9sP+hMPzmTf2mtZ+kdFps6vA8Bn01fT/8dW3NmiHjRNj/s/gYhIody92H+AvsCR+bbvB47Lt30isAyYBtyY7/nZwAfA20D9wt7n+OOP951y3nmeRpaHqSebf67neXfwHGybfeB+Ow+7g/9O1QL3/5N73cEXsVeB+x+npzv4Vxxa4P6BXOsO/gnpBe5/g0vcwT/k9C2ev2ifmZ6zMXfn/luISJkDZHgcn+XmCZieZ2YDCK2DZbHt64GF7v5ebPst4O/Ad8BrQD93n2tmtd19uZkdFTv/kgJeuyvQFWDfffc9fuHChTse4MKFzJ+bvc3MxBpVc9ir7kbcYf732w4BqlU9h3q1N5KTAwt+3HZ/nRobqVsrh+xs+Gbhtvvr1d5Ireo5rN9gfPe/XbfZv1edbGpUy2XtOuOHRdvur79HNtWq5JK1phwLfw0V93apUI5Dz9kfK6fxrCISHzOb7e6F3nJIVILoC7zm7vNj2/cD49z9s9j22+5+Yezx+cBe7j5oq9cY4+5tt/c+6enpnpGRUezxi4iksngTRKI6qScBnWOBVAOaAfPy7a9qZnVjj88CvjCzf5nZqbFzDgfWJSg2ERGJQ0I6qd19opmdaWZTY0/dDaSb2bHu/hxwFzDKzBx4z91nmNn3wCtmVhFYA9yQiNhERCQ+CRsw7+53FfD0jNi+DODUrY5fCpyTqHhERGTHaCa1iIgUSAlCREQKpAQhIiIFUoIQEZECJWQeREkxs2XATsyUA6A2sLwYw4lSqlxLqlwH6FqSVapcS1GvYz93r1PYQaU6QRSFmWXEM1GkNEiVa0mV6wBdS7JKlWspqevQLSYRESmQEoSIiBSoLCeIgVEHUIxS5VpS5TpA15KsUuVaSuQ6ymwfhIiIbF9ZbkGIiMh2KEGIiEiBymSCMLOuZjbLzGaa2YNRx1MUZtYytu73dDN7xsxK5cpBZlY1dh1PRh1LURW2HntpYmZtzOwnM2sUdSxFYWaPxda6/9jMzo06np1lZmlm9mbsOj4ys1MS+X5lMkEAWUAT4BTgJDPbJeJ4dkos7r8DZ7v7qcAKoE20Ue0cd/8TuCrqOIoq/3rsQGugt5lViDisnebuY4GhUcdRFGbWFljh7qcBZwL/Z2al9bOvHvC0uzcFrgM6JfLNSut/pCJx92GEBLEY+NLdcyIOaWdVAO5w97zFlb4Etl3rVEpSC2AwgLv/AUwFjoo0IvkWeAHA3dcAi4BS2dJ29x/c/WMzexTIAF5M5PuVyQQB4O4fAfsANfJWsitt3H2du38KYGY1gbbA6GijKvNqAUvybS8G6v7FsVIC3P1Ld88EMLPWwIxS/KUQAHe/HTgGeDiRraEykyDM7J+xe5BzzGw/AHdfCwwCToo2uh2T71qmxLarAs8Af3P3DZEGtwO2vo4UkQnkr3FTD1gaUSySj5mdDpzq7qW2n8vMGprZrgDu/i3wPeFLSUIkbEW5ZOPu9wOYWXXgDTNrFfsW0Q54J9LgdlDetQCYWRXgOeAud/8tuqh2XP7rSCF567H3yrcee99IIxLMrBlwAfC3qGMpombAycDg2GfZfu6+LFFvVmYSRB53/93MhgLTzGwjMMHdpxZyWlIys8rABKAm8GpsANNwd38+0sDKsILWY3f37EiDKuNit5DHAXOAybF/J13c/cdIA9s5Q4BBZtYJyAXuTOSbaSa1iIgUqMz0QYiIyI5RghARkQIpQYiISIGUIEREpEBKECIiUiAlCJEEihW7qxnHcSeb2aElEZNIvJQgRLbDzDrHir3trKvdfUXstfY0s6Fm9omZTTazD83sgthxK4EuRQ5YpBiVuYlyIiXFzA4kFIrLm8E/hjDjfXLsuSrAQ2b2qbvPN7MjootWZFtKECJxMLMrCOWVcwi1lW529yVmVht4Eqgf2zcGONDdewJHA7NjL3E18HhecgBw91XATfneZq2Z7VLaC8lJ6tAtJpHCHQq0BFq4++nAI0D/2L5ngWdiaw20BA7Id94ewM/5XuPjQt7nZ6B2McUsUmRKECKFywYG59VUcvcMYNfYvkruPiv2fC7wUr7zagJ5a3V8BxxfyPusI4GVOUV2lBKESOEqAZflrTxoZscQkgZAtpkdHnvegMuBvAJnqwhJAkKRtbvyL0EaWz6yn5ntEXuqJvBnQq9EZAeoD0KkcPOBX4FJZpYLLAd6xPbdAjwTG8q6AfgQqBHbt4zYYkHuvsLM2gB9zaxPvtd+2t3zFhiqE3ttkaSgaq4iRWBmtwEvufvK2PbfgB/cfbSZHQZ0d/ce232Rza/1vrufncBwRXaIWhAiRTMeeNbMVhP6Jb5199EA7r7AzOJajzo2xPXrxIUpsuPUghARkQKpk1pERAqkBCEiIgVSghARkQIpQYiISIGUIEREpED/H/+wxzuY4f+0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot accuracy曲线\n",
    "train_means = grid.cv_results_['mean_train_accuracy']\n",
    "test_means = grid.cv_results_['mean_test_accuracy']\n",
    "\n",
    "Cs = np.logspace(-3, 3, 7)\n",
    "x_axis = np.log10(Cs)\n",
    "\n",
    "plt.plot(x_axis, np.array(train_means), 'r-', label='Train')    \n",
    "plt.plot(x_axis, np.array(test_means), 'b--', label='Test')    \n",
    "\n",
    "plt.legend()\n",
    "plt.xlabel( 'log(C)' )                                                                                                      \n",
    "plt.ylabel( 'accuracy' )\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当 C 增大时，模型性能变好，但是当正则参数达到一定程度后，模型开始过拟合。此时当 C=100 时，也就是 log(C)=3，模型明显变得复杂了，训练集的正确率接近1，几乎全部分对，而模型在测试集上表现很差，这是典型过拟合现象。而在一开始，训练和测试的曲线基本重合，模型处于欠拟合状态，所以当 C 逐渐变大时，可以看见这两条曲线的正确率都在升高。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
